This Background is intended to provide the basic context of this patent application and it is not intended to describe a specific problem to be solved.
Tagging photographs with names and places is becoming common. This data is useful to users as the tags may help user recall things in the photos. Oftentimes these tags refer to the photo as a whole. For example, a photograph taken from the Empire State Building may be tagged with “Empire State Building” and “New York”. More recently, users are tagging not only the entire photograph but also subimages. In the previous example, a user may annotate the photo with the tag “Central Park,” along with an outline indicating where it is in the photo. We call such tags annotations, and describe contributions that exploit them. An annotation is a tag with an associated region of interest in a photo. The tag may be text or other media such as audio, video, etc.
Another source of tags maybe objects that can be recognized automatically in the image: A fast food restaurant logo, a known landmark (“statue of liberty”), a water body, etc. The above description may use a mix of user tagged annotations & matched points.